On Stability Region and Delay Performance of 
Linear-Memory Randomized Scheduling for 
Time- Varying Networks 
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Abstract — Throughput optimal scheduhng poUcies in general 
require the solution of a complex and often NP-hard optimization 
problem. Related literature has shown that in the context of 
time-varying channels, randomized scheduling policies can be 
employed to reduce the complexity of the optimization problem 
but at the expense of a memory requirement that is exponential 
in the number of data flows. In this paper, we consider a Linear- 
Memory Randomized Scheduling Policy (LM-RSP) that is based 
on a pick-and-compare principle in a time-varying network with 
A'^ one-hop data flows. For general ergodic channel processes, 
we study the performance of LM-RSP in terms of its stability 
region and average delay. Specifically, we show that LM-RSP 
can stabilize a fraction of the capacity region. Our analysis 
characterizes this fraction as well as the average delay as a 
function of channel variations and the efficiency of LM-RSP in 
choosing an appropriate schedule vector. Applying these results 
to a class of Markovian channels, we provide explicit results on 
the stability region and delay performance of LM-RSP. 

I. Introduction 

One key characteristic of the wireless communication 
medium is its random variations due to user mobihty and 
unpredictable changes in the radio environment. This makes the 
enduring challenge of efficient resource scheduling extremely 
difficult in wkeless networks, especially as the network size 
increases. In their seminal work [1], Tassiulas and Ephremides 
propose a throughput optimal scheduling policy, commonly 
referred to as the Generalized Maximum Weight Matching 
(GMWM) policy, that stabilizes the network for any input rate 
that is within the network layer capacity region. In this context, 
the network layer capacity region is defined as the closure of 
the set of all input rates that can be stably supported by the 
network using any possible scheduling policy [1][2][3]. 

The GMWM policy in each time-slot maximizes the sum 
of backlog-rate products given the channel states and queue- 
lengths, where this maximization can be considered as a 
GMWM problem, which can be NP-hard depending on the 
underlying interference model [4]. The complexity of the 
GMWM policy naturally has motivated many researchers to 
develop sub-optimal algorithms that approximate its solution. 
In particular, Tassiulas in a pioneering work [5] shows that 
simple randomized policies based on the pick-and-compare 
principle are sufficient to achieve throughput optimality. These 
policies in each time-slot use a randomized algorithm A to se- 
lect a candidate schedule vector that with non-zero probability 
S can be the optimal solution to the GMWM problem. Once 
a schedule vector is picked, it is compared with the previous 
schedule in terms of the sum of backlog-rate products, and 
the one with the larger sum is selected for scheduling. This 
approach, however, assumes time-invariant channels. 
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More recently, in the context of time-varying channels, the 
authors of [6] [7] have shown throughput optimality can be 
achieved if at the generic time-slot t, the randomized policy 
compares the picked schedule vector with the one used at 
the most recent occurrence of the same channel state at time- 
slot t. This proposal, therefore, requires a table with a size 
proportional to the number of channel states, and hence, its 
memory requirement is exponential in the number of data flows 
(links) N . In practice, however, mobile systems are computa- 
tionally limited and have limited memory resource. Therefore, 
any attempt to practically implement such randomized policies 
should aim at reducing both the complexity of computation and 
the required memory storage. 

In this paper, we are interested in addressing the following 
questions under the assumption that randomized policies are 
employed for scheduling: 

• How much sub-optimality in the network throughput is 
introduced by a reduced memory requirement, especially 
when the available memory storage can increase only 
linearly with the number of data flows? 

• What is the delay scaling? Does the delay increase 
exponentially as the channel states become increasingly 
correlated and the number of data flows increase ? 

Inspired by the above challenging questions, in this paper, 
we focus on a linear-memory randomized scheduling policy 
LM-RSP, which essentially follows the same pick-and-compare 
principle as the one used in the randomized policies in [5] [6] [7] 
but is generalized in the following respects. First, the update 
rule for the comparison of schedules in LM-RSP is generalized 
to be probabilistic. Second, LM-RSP uses a more general 
model for the randomized algorithm A, according to which, 
with a probability not less than a positive 6, the algorithm 
A returns a schedule vector that is within (.-neighborhood 
of the optimal solution. Considering different values for the 
pair (C, 6) allows us to study algorithms with a wide range of 
complexity levels. Note that a value of 6 less than one allows 
us to model algorithms with nondeterministic results, possibly 
those implemented in a distributed manner. In this paper, we 
limit our model to a network with N one-hop data flows, e.g, 
downlink or uplink of a cellular or a mesh network. 

Our main contribution in this paper is to analytically char- 
acterize the performance of LM-RSP in terms of its associated 
stability region and average delay in the context of time-varying 
channels. First, for general ergodic channel processes, we show 
that the stability region is a scaled version (fraction) of the net- 
work layer capacity region. Our analysis quantifies the scaling 
factor and demonstrates how it changes with channel variations 
and the computational efficiency of the randomized algorithm 
A. In addition, our analysis provides a general average delay- 
bound for the input rates strictly inside the studied stability 
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region. 

Second, to obtain more specific results, we consider an 
important class of Markovian channels where the state of 
each link is a two-state Markov chain. We assume that a link 
holds its state during one timeslot, but the state may change 
from one timeslot to another with (transition) probability r 
and independent of the states of other links in the network. 
For this simple yet worst-case modeling class of channels, r 
represents the individual link variation rate over one timeslot, 
and we show that for appropriate choice of parameters while 
the average delay is ©(p-), as r 0, LM-RSP can stabilize 
a minimum fraction — C) of the capacity region, e.g., 

when the interference is specified by the node exclusive model 
[8][4][9][10] or, more generally, by the n-hop interference 
model [4], where no two links within k hops can success- 
fully transmit at the same time. It is worth mentioning that 
this minimum fraction does not depend on the total channel 
variation rate, which approximately equals Nr for small r's, 
but, instead, depends on the individual link variation rate r. 
In addition, note that while the capacity region shrinks as the 
interference becomes more restrictive, e.g., when k increases 
in the K-hop interference model, these results indicate that 
the minimum fraction remains fixed. Another important yet 
intuitive implication of these results is that if it is possible 
to increase S, at the expense of increasing the complexity of 
algorithm A, it is sufficient to make sure that S has the same 
order as r in order to make sure that LM-RSP stabilizes a 
fraction close to (1 — C) of the capacity region. 

Our results further verify that the average delay can be 
polynomially bounded as the number of data flows increases, 
e.g., when channels are Markovian, as described earlier As 
far as we are aware, our results are the first to rigorously 
show that the delay does not need to increase exponentially 
with the number of data flows or channel correlation when 
randomized policies are used in the context of time-varying 
channels. Finally, note that in the limit of highly correlated 
channels, our results include the one in [5], which states that 
linear-complexity algorithms are sufficient to attain throughput 
optimality. 

The rest of this paper is organized as follows. In Section HH 
we review the related work. In Section |III1 we provide details 
about the system model, and, in Section |IV| we explain the 
operation of LM-RSP. In Sections [V] and [VH we state the main 
results of the paper on LM-RSP's stability region and delay 
performance, respectively. In Section [VIll we discuss important 
observations. Finally, in Section IVIIII we conclude the paper. 

II. Related Work 

Stable resource control for wireless networks first appeared 
in [1], where both the GMWM policy and the throughput 
capacity region are characterized. The work in [1], however, 
uses many simplifying assumptions for the channel and arrival 
processes. This result has been further extended by many 
researchers [1 1][2][12][6][3]. Recently, Neely and Modiano es- 
tablished the network layer capacity region for general ergodic 
channels and arrival processes [3]. In fact, the same stabilizing 
rule in [1] has been used as the scheduling component for rate 
control [13], energy optimal design [14], congestion control 
[15] [16], and the study of utility-delay tradeoffs [17]. AU of 
these papers generally assume that the scheduler has access 
to the solution of the complex GMWM problem. Another 



example of the throughput optimal control is the exponential 
rule proposed in [18]. 

One difficulty in implementing the GMWM policy is having 
access to updated queue information. In [6], it is shown that 
under general conditions, delayed or infrequent queue-length 
information does not affect the stability region. A similar result 
is shown to hold [19], when the queue-length-based scheduling 
at the base station is combined with congestion control at the 
end user, which can lead to weighted proportional fairness [15]. 
In this paper, therefore, we assume that the queue information 
is available and, instead, focus on the memory requirement and 
the complexity of the scheduling policies. 

The main difficulty in implementing the GMWM policy is 
its complexity since this policy can be NP-hard depending 
on the assumed interference model [4]. This has motivated 
many researchers to develop sub-optimal constant-factor ap- 
proximations to the GMWM policy. For instance, in [8], 
the impact of imperfect schedules is studied, where, as an 
example, a Maximal Matching (MM) scheduling algorithm is 
used to stabilize at least half of the capacity region. Due to its 
simplicity of implementation, MM scheduling has been widely 
investigated in the Hterature [20] [4] [21] [10] [22]. Despite the 
fact that these works address the issue of complexity, they are 
generally proposed for networks with time-invariant channels, 
or otherwise, do not exploit the channel correlation to improve 
the scheduling performance. 

The use of randomized policies, based on the pick-and- 
compare principle, to reduce the complexity of throughput 
optimal scheduling first appeared in [5]. In a more recent work 
[9], the authors propose distributed schemes to implement a 
randomized policy similar to the one in [5] that can stabilize 
the entire capacity region. Both policies in [5] and [9], however, 
are proposed for time-invariant channels. In the context of 
time-varying channels, other recent proposals that are based on 
the policy in [5] include [6] [7]. Although these proposals are 
throughput optimal, their memory requirement is exponential in 
the number of data flows, and thus, they may not be amenable 
to practical implementation in large networks. 

In a different context, dynamic rate allocation has been pro- 
posed in [23]. This algorithm normalizes the feasible rates by 
appropriate weights and chooses the user with the largest nor- 
malized rate. Opportunistic scheduling is used in [24] [25] [26], 
where the long term fairness is achieved by assigning offsets 
to the users' utility functions [24], or by dynamically updating 
throughput weights [25] [26]. MaxMin fair scheduling is studied 
in [27]. User level performance of channel-aware scheduling 
algorithms has been investigated in [28]. Asymptotic properties 
of the proportional fair sharing algorithms are studied in [29]. 
Our work in this paper does not consider the issue of fairness. 
Instead, we focus on throughput optimality and investigate the 
impact of channel variations on the performance of linear- 
memory low-complexity randomized scheduling policies. 

III. System Model 

We consider a wireless network with N one-hop source- 
destination pairs, where each pair represents a data flowQ. 
Associated with each data flow, we consider a separate queue, 
maintained at the source of the flow, which holds packets to 
be transmitted over a wireless link. Examples of this type of 

' Extension to multi-hop flows is possible using the methods in [1][3]. 
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network include downlink or uplink of a cellular or a mesh 
network. 

A. Queueing 

We assume the system is time-slotted, and channels hold 
their state during a time-slot but may change from one time- 
slot to another Let s(i) be the matrix of all channel states 
from any given node i to any other node j in the network at 
time t. For instance, in the downlink of a cellular network, 
s(t) will reduce to the vector of user-base-station channel 
states, i.e., s(t) — (si(t), . . . , SN{t)), where Si{t) is the state 
of the ith link at time t. Throughout the paper, we use bold 
face to denote vectors or matrices. Let S represent the set 
of all possible channel state matrices with finite cardinality 
\S\. Let Di{t) denote the discrete rate over the ith link at 
time t, and D(i) be the corresponding vector of rates, i.e., 
D(t) = {Di(t), . . . , DN{t)). In addition, let I,(t) represent 
the amount of resource used by the ith link at time t, and I{t) 
be the corresponding vector, i.e., I(t) — (/i (<),••• ,lN{t)). 
The vector contains both scheduling and resource usage 
information, and hereafter, we refer to it simply as the schedule 
vector. Details for the selection of I{t) are provided in Sec- 
tion HV] Let T denote the set containing all possible schedule 
vectors, with finite cardinality \T\. 

Note that the exact specification of the schedule vector I{t) 
is system dependent. For instance, in CDMA systems, it may 
represent the vector of power levels associated with wireless 
links, or when the interference is characterized by the K-hop 
interference model [4], the vector I(t) can be an activation 
vector representing a sub-graph in the network. 

Since transmission rates are completely characterized given 
the channel states, the schedule vector, and the interference 
model, we have 

D(i) =D(s(t),I(i)). (1) 

We assume that transmission rates are bounded, i.e., for all 
s G S and 1 E T, 

Di{s,l) < D^ax, 1<1<N, (2) 
for some large Dmax > 0. 

Let Ai{t) be the number of packets arriving in time-slot 
t associated with the ^th link (or data flow), and A{t) be the 
vector of arrivals, i.e., A{t) — {Ai{t), ■ ■ ■ , ^jv(t)). We assume 
arrivals are i.i.dQ with mean vector 

E[A(i)] = a = (ai, . . .^un), 
and with finite second moments: 

E[Af{t)] < 1 < ; < ^, (3) 

for a suitably large Amax- Assuming || • || represents the 
Euclidean norm of a vector, we define E[||A|p] as 



E[||A|p]^E[||A(<)|| 



N 

E 

1=1 



E[Af(<)]. 



(4) 



Since the arrival process is i.i.d., we see that E[|| A||^] is well- 
defined and is independent of t. By Markov's inequality, we 
have the following fact. 

Fact 1. For any given positive e, there exists a sufficiently large 
such that for A > A^, we have p{\\A{t)\\ > A) < e. 

^This assumption is made to simplify tlie analysis, and the extension to 
non-i.i.d. arrivals is straightforward. 



Finally, let X(t) = {Xi{t), . . . , XN{t)) be the discrete 
vector of queue-lengths, where Xi{t) is the queue-length 
associated with the /th link. Using the preceding definitions, 
we see that X(t) evolves according to the following equation 



X(t + 1) = X(i) + A{t) - D(t) + U(i), 



(5) 



where U(t) represents the wasted service vector with non- 
negative elements; the service is wasted when in a queue the 
number of packets waiting for transmission is less than the 
number that can be transmitted, i.e., when Xi{t) < Di{t) for 
some 1,1<1<N. 

B. Channel State Process 

We assume the channel state process is stationary and 
ergodic. In particular, similar to [3], we assume for any given 
positive e, there exists a Ki ,, such that for K > Ki ,: regardless 
of the system state at time t denoted by 

Y(<) = (X(t),I(i),s(t)), 



we have 

E|-(s)-e[1eis(* 



K-1 



+k)=s 



Yit) 



ses 



k=0 



(6) 



where 7r(s) is the steady-state probability of the channel at 
state s, and Ig is the indicator function for the event e. This 
inequality simply states that the expected value of the average 
number of visits to a given channel state converges to its steady- 
state probability, and the sum of the absolute value of the 
differences, over all possible states, vanishes for sufficiently 
long time-intervals. The above further implies that there exists 
a K^'^} such that for K > K^2.h we have 



K-l 



l^k=Q iK-k k=0 



E|vr(s)-E[ 

where is 

numbers with the property that lim^ 

'(7) 



7x-fcls(t+fc)=s|Y(t)] 



< e, 



an increasing sequence of positive real 

) li = 7oo < 00. Note 



that e i'^ general depends on the sequence {7i}^o- 
particular interest is the case where 7^ — J^j^ii^ — Sy , which 
defines the sequence for the rest of the paper Examples 

of processes that satisfy the above inequaUties include but are 
not limited to Markov chains. 



IV. Scheduling Policy 

In this section, we elaborate on the statistical structure of the 
algorithms that provide sub-optimal solutions to the GMWM 
problem, and describe the operation of LM-RSP that uses 
these algorithms. We start by providing a brief overview of 
the network layer capacity region and the precise definition of 
the GMWM problem. 

In [1][2] and recently under general assumptions in [3], it 
has been shown that the capacity region is given by 



r = V7r(s) Convex-Hull{D(s,I)|I e I}. 



ses 



(7) 



Moreover, it has been shown that the GMWM policy is 
throughput optimal in that it stabilizes the network for all 
input rates that are strictly inside T [1][2][3]. This policy at 
each time-slot sets X = X(i) and s = s{t), and uses the 
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schedule vector I* (i) that is argma^ to the following GMWM 
optimization problem: 



N 



max XiDi{s, I), subject to I € J. 



(8) 



(=1 



Hence, the GMWM policy uses a schedule vector that maxi- 
mizes the sum of backlog-rate products. However, note that the 
optimization problem given in ([8]) can be NP-hard [8] [4]. We 
therefore consider a policy based on randomized algorithms 
that can provide approximate solutions to the optimization 
problem in ([8]l. In the following, we first elaborate on the 
structure of the considered sub-optimal algorithms. 

A. Sub-optimal Algorithms Approximating GMWM Problem 

In this paper, we assume that there exists a randomized 
algorithm A, either centralized or distributed, that at each time- 
slot t provides the network with a candidate schedule V{t) 
from the set T. We use superscript r to emphasize that V{t) is a 
candidate schedule vector selected by the randomized algorithm 
A. In general, the distribution of the selected schedule V{t) 
depends on X(t) and s{t). We define /ix(t).s(t)(') '■^ 
distribution for Note that the policy developed in this 

paper does not need the knowledge of A*x(t),s(t)(')' ^^'^ only 
requires the algorithm A to satisfy Properties [T] and |2] as will 
be discussed shortly. 

Given X and s, let the optimal schedule and the optimal 
rate associated with the GMWM problem be I* (X, s) and 
D*(X,s), where 

(9) 



r(X,s) = argmaxXD(s,I), 
lex 



and 



D*(X,s) =D(s,r(X,s)). (10) 

Note that in the above, XD(s,I) denotes the scalar product 
of the vectors X and D(s,I), and for simplicity, we have 
dropped the dot operator. In the rest of the paper, we use the 
same method to denote scalar products of vectors. In addition, 
note that by ([U, D(s, I) is the rate vector corresponding to 
the channel state s and the schedule vector I. We assume the 
algorithm A has the following property. 

Property 1. There exist a constant C < C < 1> ^^^d a 
constant S, d > 0, such that, for any given X and s d S, with 
probability at least 5, the algorithm A finds a candidate vector 
V that satisfies the following: 

XD(s,r) > (1 - C)XD*(X,s). (11) 

This property simply states that the selected schedule I*" 
with probability at least 6 is within ^-neighborhood of the 
optimal solution in terms of the backlog-rate product. We 
can consider this property as a generalized version of the 
ones in [5] [6] [8] [7] [9], modeling the sub-optimality of the 
algorithm A through the introduction of the pair {(, S). The 
following further details the structure of the algorithm A by 
stating a property for the distribution set {/ix s('); s G 5, X G 
({0}UZ+)^}. 

Property 2. Consider two queue-length vectors Xi and X2, 
and suppose |jX2 — Xi|| < C /or a given constant C > 0. For 

^If there are more than one schedule vector maximizing the summation in 
{8), we define the argmax to be any of such schedule vectors. 



any given positive e, there exists a sufficiently large ^ such 
that if ||Xi|| > B'(^, then 

^|A*x.,s(r=I)-A^x,,s(r = I)| <e, 
lei 

for all s G iS. Moreover, for any two given Xi and X2, ;/ 
X2 = /3Xi, for some /? > 0, then, for all s ^ S and I G I, 

Aix„s(r = I)-A^x,,s(F=I). 

This property states that the distributions for I'' are almost 
the same when two queue-length vectors are close and large. 
This property also states that the distribution is exactly the 
same if two queue-length vectors differ only by a multiplicative 
scalar factor. These statements may naturally hold since the 
objective function in (O is a continuous function of X, and 
assuming ||X2 — Xi|| < C, for any s and I, by (|2]l, we must 
have 



|X2D(s,I) -XiD(s,I)| < ViVCA„,,. 

Hence, finite changes in the queue-length vector have a finite 
impact on the backlog-rate product. This and the fact that 
XD*(X,s) linearl>0 increases with ||X|| suggest that the 
impact when normalized to XiD*(Xi,s) can be arbitrarily 
small if ||Xi|| is sufficiently large. We therefore expect the 
algorithm A, for each pair (s, I), to see similar normalized 
valued of the backlog-rate products corresponding to Xi and 
X2. Hence, for given Xi, X2, and s, the algorithm is expected 
to assign similar probabilities for F = Iwhen ||X2— Xi|| < C, 
and ||Xi|| is sufficiently large. In the case where X2 = /3Xi, 
the backlog-rate product corresponding to X2, for all s G 5 and 
I G I, is a /3-scaled version of the one for Xi. Therefore, we 
expect the distribution for I'' corresponding to X2 to be exactly 
the same as the one corresponding to Xi. Having detailed the 
structure of the algorithm A, we next focus on the operation 
of LM-RSP 

B. LM-RSP' s Operation and Scheduling 

We start by defining several useful functions. Let D'^(t) and 
D (< — 1) be defined as 

and 

D'(t-l)=D(s(t),I(i-l)), (12) 

respectively. According to the above definitions, we see that if 
the network uses the candidate schedule Y{t) at time-slot t, the 
resulting rate vector will be D*" [t) whereas if the network keeps 
using the schedule vector of the previous time-slot, l{t — 1), 
D'(t — 1) will be the rate vector (at time-slot t). In addition, 
let if{t) be defined as 

X(0(D-(t)-D'(t-l)) 



(13) 



max(X(t)D'-(t), X(t)D'(t - 1)) + a||X(i)|| ' 
where a is a positive but otherwise arbitrary constant. Later in 
this section, we elaborate on the motivation to consider a non- 
zero value for a. Based on the above definition, tp(t) measures 
the normalizeJ^ improvement in terms of backlog-rate product 
when the candidate schedule V{t) is preferred over I(t — 1) at 
time-slot t. We define ip{t) = if X{t) = 0. 

"^Later, we show that XD* (X, s) > — ^ where v is a positive system 
dependent constant as defined in (19). 

'Here, for a given X, the product is normalized to XD*(X, s). 
'Here, by normalization, we mean division by a well-defined function. 
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We are now ready to describe the operation of LM-RSP. We 
assume that the policy, either centralized or distributed, takes 
as the input the vectors I(< — 1), X(t), and s{t); and using the 
algorithm A, updates the schedule vector I{t) according to the 
following: 

• Using the algorithm A, the policy selects 1(0) according 
to the initial X(0) and s(0). 

• For t > 0, it determines I(t) through the following steps. 

- First, the policy uses the algorithm A to select V{t) 
according to X(t) and s{t). 

- Next, it updates I{t) according to the following rule: 

r(f\ = / ^'^^ probabiHty /(^(i)) 

^ ' \ I{t-1) otherwise 

where ip{t) is defined in ( flJl l, and / : (— 1, 1) — s- [0, 1] 
is a non-decreasing continuous function. 

We assume the f{(p) — 0.5 is an odd function of ip, and f{(p) 
has the property that 

\ 0, < -p ' 

where < p < 1. In the rest of the paper, we assume the 
function f{'p) is linear in the range [— p, p] and f{'p) = 0.5+^ 
for \ip\ < p; we leave finding the optimal f{(p) as an interesting 
open problem for future research. Considering the definition of 
(p{t) and the properties for /(•), it is easy to see that w.p. 1 

X{t){T>{t) - (1 - p)D'(t - 1)) > -pa||X(i)||. (14) 

Note that a similar inequality holds by replacing D'(t — 1) with 
D'^(t). 

The above description suggests that the memory requirement 
of the policy is linear in the number of data flows N. This is 
because the only past information required to update I{t) in 
each time-slot is the vector 1) whose size is proportional to 
N. As mentioned earlier, other similar proposals in the context 
of time-varying channels [6] [7], store one vector for each 
possible channel state. However, the number of states increases 
exponentially with N, which implies that these proposals 
require an exponentially increasing memory storage. 

One subtle point in the design of the policy is the introduc- 
tion of tf{t) and f{(p{t)) in the update process of I(<). These 
functions allow LM-RSP to take soft decisions when com- 
paring two different schedules, generalizing similar previous 
approaches in [5] [6] [7] [9]. Specifically, these functions enable 
LM-RSP to probabilistically choose either of the schedules, 
I(t — 1) or l^it), according to the value of <p(t); a larger 
positive (p{t) implies that selecting T'{t) as the schedule vector 
results in a larger backlog-rate product, which according to the 
monotonicity of /((p), increases the probability of selecting 
V{t). Similarly, a smaller negative ip{t) increases the chance 
of selecting I(t — 1) as the schedule vector for time-slot t. 

However, a mere generalization is not the main motivation 
for the introduction and use of ip{t) and f{(f{t)). The main 
motivation is to make the distribution of I{t) continuous with 
respect to X(i) for large ||X(t)||, and thus, is analysis-inspired. 
More specifically, we have the following fact, which results 
from the continuity of /(•), the assumption that a > 0, and 
the fact that the distribution of I{t) is completely determined 
by the values for I{t - 1), X{t), r{t), and s{t). 

Fact 2. Suppose two vectors Xi and X2 are given such that 
11X2 — Xijl < C. For any given positive e, there exists a 



sufficiently large B2 f such that if |lXi|l > B2^, then for all 
Y{t) £ I, I{t - 1) e I, s{t) e S, and I G {V{t),I{t - 1)} 
the following holds: 

\pim = i\x{t) - X2, s{t),r{t),i{t - 1)) 

-p{m - I\X{t) - Xi,s(i),rW,I(i- l))l < e- (15) 

One important point that should not be buried under the 
main motivation is that introducing ip{t) and /((p) allows us to 
consider and embed in our model the efficiency of the update 
rule when it is implemented distributedly. The update rule in 
its original form [5] [6] [7] simply takes a decision based on 
only the sign of (p{t) and uses the schedule vector, either I{t — 
1) or T'it), with the larger backlog-rate product. When the 
update rule is implemented in a distributed manner, however, 
the estimates of X{t), X(t)D''(t), and X{t)'D'{t - 1) can be 
erroneous. These inaccurate estimates may make the computed 
(p(t) and, in particular, its sign to be different from their actual 
values. Hence, the sign of (p{t) alone may not be sufficient 
for comparison purposes. This suggests to use the value, and 
not only the sign, of (p(t) to evaluate and select the vectors 
l{t— 1) and V'it). In this case, while the choice for a positive 
a is arbitrary, the value of p may be adjusted to account for 
the extent within which the estimates are inaccurate. Note that 
when LM-RSP is implemented in a centralized manner, the 
choice for both a and p is arbitrary as long as a > and 
< p < 1. 

C. Complexity of LM-RSP 

As mentioned earlier, at each time-slot t, the GMWM policy 
solves the optimization problem in ^ to find I* (X(t), s(t)) 
as the schedule vector This problem is in general non-convex 
due to physical layer interferences, and can be NP-hard [4]. In 
contrast, LM-RSP assumes access to the algorithm A whose 
complexity depends on the value of the pair 5). For instance, 
when (5=1 and C = 0, the algorithm A always returns the 
optimal solution, and when 5 = and C = 0, the algorithm 

simply selects schedule vectors with equal probabilities. It is 
easy to see that the latter case, with purely random selection, 
can make the complexity of LM-RSP linear in N. This special 
case is attractive mainly from a theoretical point of view since it 
achieves throughput-optimality in the limit of highly correlated 
channels, despite an exponentially increasing delay due to the 
exponentially decreasing S with the number of data flows (see 
Corollaries |2] and |6]l. More interesting examples are discussed 
in [30], where a distributed algorithm is developed with a time- 
complexity of a; • c- e~ ^ log ^ log N, where c is a constant, and 
x> 1 and e > are tuning parameters such that ^ = 1 — 
and (5 = 1 - N^-"". 

V. LM-RSP Stability Region 

In this section, we study the stability region of the network 
under LM-RSP by first providing several key definitions. 

A. Key Definitions 

For notational convenience, in the rest of the paper, where 
appropriate, we use subscripts also to show dependencies on 
time; hence, e.g., we have Xt = X{t). Let T(Xt) be defined 
as 

T(XO =E[XtD*(Xt,s)], (16) 
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where the expectation is over the steady-state distribution of the 
channel process, and D*(Xf,s) is defined in ( fTOb . Based on 
this definition, T(Xt) denotes the expected value of the max- 
imum backlog-rate product and, thus, is the expected backlog- 
rate product if the GMWM policy is used, and the queue-length 
vector is fixed at X = Xj. The quantity T(Xt), therefore, 
can serve as a benchmark to measure the performance of sub- 
optimal policies. 

In our analysis, we often encounter distributions and ex- 
pected values of random variables where after a particular time 
t, queue dynamics are ignored. To make this notationally clear, 
suppose a rv. Z is given, which can be a function of the 
channel process and the selected schedules. We define [Z] 
and pyit{Z = z) as the expectation of the r.v. Z and the 
probability that Z ~ z, respectively, given the hypothesis that at 
any time where t' > t, the policy updates I(t') by assuming 
X(t ) = X(t). In other words, these notations emphasize that 
after time t, the policy makes decisions based on the old queue- 
length information at time t. These notations further assume 
that the updated channel state information s{t'), t' > t, is 
available. Note that without the above hypothesis, both updated 
queue and channel state information are available for LM-RSP 
to update the schedule vector. 

Having introduced Ext [ 

(i-p)D;+„j|Yi] 



we define ^I^y^ as 



(17) 



By the definition in Dj is the rate vector corresponding 
to the schedule vector in the current time-slot, I(i), and the 
channel state in the next time-slot, s{t + 1), implying that 
'Dt+m — (1 ^ _!_,,„ for small values of p approximately 
shows the changes in the rate vector 'Dt+m due to channel 
variations. Note that the sequence of Dj+m's not only depends 
on a particular realization of the channel states but also on the 
randomized algorithm A that finds the candidate schedules. 
Therefore, when K is large, ^I^y^ measures the relative changes 
in the backlog-rate products, due to channel variations, over a 
long horizon, while implicitly embedding the effects of the 
algorithm A. This implies that ^I^y can be used as a measure 
for the channel correlation since ^1^:^^ becomes small for small 
values of p if the channel states are highly correlated. It is 
important to note that in the definition of ^I^ y^ , the expectation 
is of the type ExJ ], and hence, queue variations after time t 
do not affect ^E* y^ • In addition, note that V^Yj not only depends 
on K and Yt but also implicitly depends on t. 
Similarly, let 



Z]m=0 [XtDt+„ I Yt 



" KTOQ ■ ^^^^ 

This definition introduces <I>Yj as the time average of backlog- 
rate product normalized to the benchmark T(Xt). Hence, we 
can use <I>y as a measure to compare the performance of LM- 
RSP with that of the GMWM policy. 
As for one other definition, let 

(19) 

ses 

Thus, u is the minimum of the average maximum transmission 
rate for individual links, over all links in the network. This pa- 
rameter is a fundamental property of the system. It immediately 
follows that if for every link, there is at least one state in which 
the transmission rate is non-zero, then v > Q, which is assumed 



v — mill > 7r(s) maxl?;(s, I). 
KKN lei 



throughout the paper One importance of this parameter is that 
we can obtain a lower bound for T(Xt): 

T(Xt) > max Xi(t) > ^||Xt||. (20) 

^ ^ ~ 1<1<N ^ ' - //V 

Now, we define a key parameter B that represents the fraction 
of the capacity region F that can be stabilized by LM-RSP. 
Specifically, we define Q as 



= liminf inf max(l — C' ^ 
where 



1 



^Yt 



Npct 



1>yJ, (21) 



We assume 



C =(i-(i-p)(i-C)). 

< 1 or, in other words, 

pa. < 



(22) 



N 



(23) 



This assumption can be, in general, a necessary condition for 
the positivity of the first argument in the max operator of ( |2TI ). 
By the definition of the liminf, we have the following fact. 



Fact 3. For any positive e > 0, there exists a sufficiently large 

HVpa 



Ke , such that for all Yt and K > , the following holds: 



max(l — C' — 



1-5 



' ^Yt 
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To shed light on the properties of the parameter Q, and also 
to consider an important special case, suppose the channel 
process is Markovian. According to our assumptions, there is 
a finite number of channel states and schedules. Therefore, 
given the hypothetical condition that the queue-length vector 
is frozen at Xt after time t, as assumed in the definition 
of 5'yj and ^Yt' '■^^ joint process of rates and the channel 
states will be a Markov chain with a finite number of states. 
If the joint process has a single communicating class and is 
aperiodic, e.g., when the randomized algorithm selects any 
schedule with a positive probability, and the channel process is 
irreducible and aperiodic, then the joint process will be positive 
recurrent and will have a steady-state distribution. Hence, as 
m oo, Dt+m weakly converges to a random vector D whose 
distribution depends on the channel distribution, the algorithm 
A, the update rule, and the given vector X = Xt. A similar 
discussion also holds for Dj_|_„j. Therefore, in the limit of large 
K, both ^^Yj and $Yt become independent of initial I{t) and 
s{t). In particular, assuming X = Xt, as if — + cx3, we have 
.K . .„oo A .„oc^ E[X(D - (1 - p)Dy 



Yt 



4»= 



Yt 



and 



K 



— ^Yt — 



T(X) 
EfXDl 



(24) 



(25) 



-Yt ■ -Yt Y(X) 

where in the above expressions, expectations are taken with 
respect to the distributions for D and D', and by using 
and 'I'x 1 we have misused the notation to emphasize that ^^y^ 
and $Yt depend on Yt only through X = Xf. 

As one other observation for this special case, note that by 
the Markovian nature of the channel process, ^'y^ and $Yt 
become independent of t when Yt is given. This independence. 
Property [2] and the update rule further imply that the distribu- 
tions for ^fy^ and $Yt do not depend on ||Xt||. Therefore, in 
this case, despite the fact that the vector Xt is discrete, for the 
purpose of taking the inf over Yj in the definition of 9, we 
can replace Xt, as one element of Yt, with In addition. 



since the inf is taken over all possible fs. 



Xt 

l|Xt 



can take all 
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possible directions and can be any unit vector in the limit of 
large t. Hence, by (l24b and dZSb . for Markovian channels, we 
must have 

e= inf max(l-C' - ^-^*x - ^^?^,*x)- (26) 
X:||X|| = 1 ^ ^ 6 ^ 5v ^' 

B. Theorem on Stability Region 

The following is the main result of this paper on the stability 
region of LM-RSP. 

Theorem 1. Suppose the mean arrival rate vector a lies strictly 
inside 9T, where 6 is defined in ( 1211 ), and OT is a region that 
contains 9-scaled of all rates in T, i.e., OT — {ae] 3a G F : 
ag = 9a\. We have the following: 

(a) There exist non-negative constants /3s, i'* such that 

a=^7r(s)^/3s.iD(s,I), (27) 

and 

e = 9 — max > /3s i > 0. 

(b) Under LM-RSP, the system described in Section \in\ is 
stable in the mean, i.e., 

1 ^ 

limsup— -^E[ llXtll ] <oo. 
Proof. The proof of the theorem is given in the appendix. □ 

C. Insights into the Region OT 

Here, we discuss several practical implications of the above 
theorem by first focusing on general channel processes and 
then considering an example of Markovian channel states. 

1) Insights Assuming General Channel Processes: As the 
first point, the theorem suggests that a scaled version of the 
capacity region T can be supported by LM-RSP. The theorem, 
moreover, shows that the scaling factor is 0, which by definition 
depends on the limiting behavior of the policy when queue 
dynamics are ignored. Recall that is a function of and 
$Yt ■ As explained in Section IV-AI y^ measures normalized 
rate changes due to channel variations over time when queue 
variations are ignored after time t. Since, for a given time- 
slot, the policy updates the schedule vector by comparing a 
candidate schedule with the one used in its previous time-slot, 
we expect that large channel variations, and thus, large ^'yj' 
negatively affect the update process, and hence, 9. On the other 
hand, as explained earlier, for a given Yf, $Yt is a measure to 
compare LM-RSP with the GMWM policy; a larger and close- 
to-one value for <i>Yj indicates that LM-RSP uses schedule 
vectors with similar backlog-rate products to the ones resulting 
from the GMWM policy, and a smaller and close-to-zero value 
for <i>Yj indicates that LM-RSP is performing poorly compared 
to the GMWM policy. As a result, we intuitively expect to 
be as large as the least value of $Yt for large A"s, which is 
the inf (over Yt) of 5'yj for large K's. The expression for 9 
exactly reflects these observations. 

We also observe that the parameters a and p can directly 
affect 9 through the term and indirectly through the 

terms ^I^y^ and $ y^ ■ Recall that these two parameters must be 
positive for continuity purposes but, otherwise, can be chosen 



arbitrariljQ. Note that ^ is a given parameter, and u can be 
estimated readily. Hence, we might naturally try to choose a 
and p such that is arbitrarily small. In fact, assuming 

9iim exists, where 

^h™ = lim 9, (28) 

we can ensure 9T contains a region arbitrarily close to 9ii„iT 
by assuming sufficiently small values for p and a, which gives 
rise to the following coroUary. However, note that, as shown 
later, the delay bound can increase proportionally with 

Corollary 1. For any input rate strictly inside the region 9iimT, 
the parameters a and p can be chosen sufficiently small such 
that the the system described in Section |^ is stable under 
LM-RSP. 

We now consider the effect of channel variations on 9. 
Suppose channel states are highly correlated. This implies that 
D^+„ ~ Df+,„. Since by definition, E[XD] < T(X), from 
( [TtI i. we have ^^y^ < p' , where p' ~ p. Assuming that p 
and a are sufficiently small and using ( |2TI) . we have that 
6* > 1 — C' — 1 — C- It is interesting to see how the presence 
of the term in 9 is canceled by the channel correlation. 
Note that the term is the average number of times that 
the algorithm A must be run before ( fTTT ) holds for a fixed 
X and s. The effect of this term is reduced when channel 
correlation is high, which manifests itself in a small "^^^ ■ We 
can also easily prove that if the candidate schedule returned by 
the algorithm A is used without any comparison in each time- 
slot, in general, the scaling factor becomes (5(1 — C)- Therefore, 
we see that LM-RSP improves the capacity region scaling from 
(5(1 — to at least 1 — C and, exploiting channel correlations, 
reduces the uncertainty of the randomized algorithm A in 
selecting a candidate schedule satisfying ( fTTT i. A special case 
is where C = 0, which implies 6* > 1 in the limit of p 0, 
and thus, 9 — \. Since 9 = \ means throughput optimality, 
we conclude that simple linear-complexity algorithms, see the 
discussion in Section IIV-CI are sufficient to attain throughput 
optimality arbitrarily closely, reminiscent of the results in [5]. 
We summarize the above in the following corollary. 

Corollary 2. The stability region OT contains the region (1 — 
C)r and 9iim > 1 — C ff^^ limit of highly correlated channel 
states and small p and a. In particular, when C = 0, in the 
limit, the region 9T expands to the capacity region T, and LM- 
RSP becomes throughput optimal. 

2) Insights Assuming Markovian Channel Processes: Our 
discussion so far considers networks with general channel 
processes. In the following, to obtain specific results, we 
focus on an important class of Markovian channels and well- 
investigated interference models. Suppose the channel states of 
wireless links are independent. Furthermore, suppose the state 
of each link is a Markov chain with two states, namely the g 
state representing the "good" state and the b state representing 
the "bad" state. We assume that the state of a link in each 
transition can take a different value with probability r. Hence, 
r may represent individual link variation rate over one time- 
slot. As the worst-case scenario, we assume in the b state the 

^As discussed in Section |IV-B| if the update rule is implemented distribut- 
edly, p may be used to model the inefficiencies in implementing LM-RSP. In 
this case, the choice for a positive a is still arbitrary. 
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transmission rate is zero. We do not impose any assumption, 
other than positivity, on the transmission rate in the g state. 
Therefore, when two links are in their g states, they can see 
possibly different but non-zero transmission rates. 

As for the interference, we consider the classic node- 
exclusive interference model [8] [4] [9] [10], where a node can 
only send to or receive from one other node at any time. This 
interference model motivates us to view the network as a graph 
G(V, E), where V is the set of users and E is the set of all links 
in the network. Given this graph, a valid schedule is a matching, 
where a matching is a set of edges no two of which share a 
common vertex. We assume the algorithm A always returns a 
matching with respect to G, ensuring that the schedule vector It 
is also a matching. Note that our discussion here easily extends 
to the more general K-hop interference model [4], according to 
which, no two links within /t hops can successfully transmit at 
the same time. 

Having defined the channel and interference models, we now 
derive an upper-bound for ^'yj- Recall that ^'y^ is almost the 
time average of Ext [X* {T>t+m - (1 - p)'^'t+m)\^ t] taken over 
m, < m < K — 2, and normalized to T(Xf). Consider the 
time t + m, and suppose the lih link is in its g state and is 
scheduled to receive non-zero transmission rate. In the next 
time-slot, with probability r, the state of this link changes to 
the b state, whose definition implies that D[{t + m) = 0. On 
the other hand, with probability 1 — r, the link stays in its g 
state. Since by definition, Df_|_„ is the rate at time t + m + 1 
but with the schedule vector It+m used at time t+m, and since 
by assumption schedule vectors are matchings, we see that no 
links that can possibly interfere with the Ith link are scheduled 
at time t+ni+1. Hence, when the kh link stays in its g state, we 
must have D'i{t+m) = Di{t+m). In the case where the Zth link 
is in its b state at time t+m, or not scheduled at time t+m, then 
Di{t + m) — 0. Considering all the above cases for link I, it is 
easy to see that lExt[^i(i)(A(i+m)-(l-p)L';(i+m))|Yt] < 
Ex, [Xiit){r + p{l - r))Di{t + m)\Yt]. 

Since the above discussion holds for all links, we have that 

ExJXt(Di+™-(l-p)Dj^.„J|Y,] 

< (r + (1 - r)p)lEx, [XtDt+,„|Yt], 

which implies that ^I^y^ < ('^ + (1 ~ ''^)p)^^^,- Using this 
inequality, the definition of 9, and the fact that max(a — 
bx,x) > j-^, we can show that 



> 



i-C 



1+ l_A(^ + (l_^)p)■ 



The term in the right hand side of 



(29) 



can be made 



arbitrarily small by choosing the policy parameter a sufficiently 
small, which, as we show later, comes at the price of increasing 
the delay -bound proportionally to ^. Summarizing the preced- 
ing discussions, we have the following corollary. 

Corollary 3. Suppose the state of each link is a two-state 
Markov chain with transition probability r and independent of 
the states of other links in the network. In addition, suppose the 
interference can be modeled by the node exclusive interference 
model or, more generally, by the n-hop interference model. 
Finally, suppose the algorithm A always returns a matching 
(or a K-valid matching) with respect to the network graph G. 



Then, for any input rate strictly inside Omin^, where 

there exists a sufficiently small a such that the network is sta- 
bilized under LM-RSP. In other words, the region 6T contains 
the interior of the region 9minX in the limit of small a's. In 
addition, when p <^ S, we have 

1-C 



1 



The corollary essentially states that a fixed fraction of the 
capacity region F, regardless of the number of data flows N, 
can be stabilized by LM-RSP given that the pair {(, 6) and 
the rate r are fixed. Furthermore, it remarkably states that the 
total channel variation rate 1 — (1 — r)^, which is close to 
Nr for small r's, does not appear in the lower-bound fraction 
9rnin, and what appears is the individual link variation rate r. 
As the last observation, note that the more restrictive is the 
interference model, i.e., when k becomes large in the K-hop 
interference model, the smaller is the region F. However, the 
corollary assures that for a given {(, 5) and r, the lower-bound 
fraction 9min is not affected by the choice of k, and thus, a 
fixed fraction of F can be stabilized no matter how restrictive 
is the interference. 

VI. LM-RSP Delay Performance 

In this section, we study the delay performance of LM-RSP. 
We start by introducing a few important parameters. 



A. Convergence Parameter K and Norm Lower-Bound B 

Here, we introduce two key parameters that play a central 
role in the delay analysis. The first is that essentially is 
a function of how fast channel states converge to their steady 
states, where the variable e is used to measure the closeness of 
the input rate to the boundary of the region 9T. In our analysis, 
determines the number of steps used in the Lyapunov drift- 
analysis. To formally define K^, suppose a positive e is given. 



and let ei 

as 



6 D„ 



N 4' 



, 62 = f £1, and £3 



We define K. 



= 2max(Xi,ei,i4:, 



(7) 



6^1 ^W) 
V e ^ 



(30) 



£3 



where i^i ^^ and defined in Section UlI-BI and K, 

is defined in Fact [3] 

The second parameter is , which acts as a lower-bound 
for the norm of ||Xt||, above which the Lyapunov drift becomes 
negative in our analysis. More specifically, if ||Xt|| > B^ , 
then within the K timeslots after time t, the inequalities in 
Property |2] and Fact |2] hold with high probability. To formally 
define B^ , suppose for a given K and a positive e, and £5 
ai-e defined by £4 = — "^IW ^5 ~ "^^^ = 
Af , + VND„iax, where A^^ is defined by Fact[T] We define 
as 



^£4 



(31) 



c 



where C = KA^^, and B^ 
Property |2] and Fact |2] respectively. 



and i?2^g are defined by 
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B. Big O Notation 

As a notational convenience in our following analysis and 
discussions, we use the big O notation with multiple variables. 
In such cases, we assume the ordinary big O notation holds 
individually for each present independent variable as it takes its 
limiting value. In particular, we consider the scaling behaviors, 
as oo, p ^ 0, a ^ 0, (5 0, C ^ 1, or e 0, where e 

is defined in Theorem [T] 



C. Theorem on Average Expected Queue-Lengths 

The following is the main theorem on the average expected 
queue-lengths. 

Theorem 2. Under the assumptions in Theorem\l\ the expected 
queue-lengths satisfy the following: 

T N 

J2 Xi^ limjup ^ ^ [ ^ E [X, it)] 



KKN 



t=0 1 = 1 

A^„ „, .^^KN^^ 



<B(V]^+2-||a||)+0(^), 



where v is defined in ( I19l l, e is given by Theorem\J] d is defined 
in Property \I\ K = K^, and B = Bf. 

Proof. The proof of the theorem is given in the appendix. □ 



To gain insights into the delay performance of LM-RSP 
using the above theorem, we need to study the properties of 
the parameters K^^ and B^ , where e is defined in Theorem [T| 
Note that this value of e is used to determine e^'s, 1 < i < 5, 
defining and Bf^ . We start by considering the definition 
of B^ given in ( l3Tl i. implying that B^ > -B^^j, ■ It is easy to 
see that since /(•) is linear in the range [~p,p], we can set 
^( ^"^aeT"" + 1). It is also easy to verify that this 



form for B2g^ is indeed necessary when constant multiplicative 
factors are ignored. Hence, by the definitions for £4 and £5, and 
that C = KA^^, all given in Section FVI-AI we have 



-)• 



(32) 



By we have K > 



paSe 

— , and by ( |23] l. we have pa < 



■^=. Using these inequalities, the equality ( [32b . and that Bf^ > 
B2^^, it is easy to see B{\/N + 2-^||a||) dominates the term 
^(^Te~)' therefore, have the following corollary. 

Corollary 4. The dominant term in the average queue-length 

isB{VN + 2^M\). 

Note that the dominant term implicitly depends on K — 
through the term B = B^ . Therefore, to have a specific 
bound on the average queue-lengths, we need to also study K^. 
By definition, depends on Ki^^-^, K^}^, and KfJ, which 
can be considered as convergence rates. The first two rates are 
essentially the convergence rate of the channel process to its 
steady state. The third rate , depends on both the channel 
convergence rate and the update policy. Hence, towards having 
a specific average delay-bound, we need to focus on a particular 
channel model, as discussed next. 



D. Specific Delay Bound for Markovian Channels 

As a special example, suppose the channel state is a Markov 
chain as described in Section IV-C.2I where the state of each 
link is a two-state Markov chain with transition probability r 
and independent of states of other links in the network. The 
following is the key lemma [31] that we use to study the 
convergence rates of Markov chains. 

Lemma 1. Suppose a Markov chain is defined on the finite 
state space X with transition probabilities P{x, y), where 
X X and y X. Let tt^ be the distribution after k 
transitions given an initial distribution ttq. Then, given any 
initial distribution ttq and stationary distribution tt, we have 

J2 kfelx) - ^(x) I < 2(1-/3) L^J, 
xex 

where fco is a positive integer and 

^mijiP*'-''(x,y), 

where denotes the transition probability after kg transi- 
tions. 

(6) 

We first concentrate on KL . Since the channel is Marko- 

(0) 

vian, by the definition of Ke^ and the discussion leading to 
(l24l) and (l25b . we see that K^s^ depends on how fast '^Yt 
and <I>Yj converge to ^I^y^ and $Yt' respectively. Recall that 
in 'J'yj^ and $Yt queue dynamics are ignored, and I{t + k), 
k > 0, is updated by setting X.{t + k) — X(t). Using this along 
with the discussion leading to (l24l i. we see that the process 
{(st+fc, li+fc), fc > 0} is also a Markov chain on the space 
S X I. Our goal is to find an appropriate value for f3 to apply 
Lemma [T] to this Markov chain. 

Consider the l± link whose state si is by itself a two-state 
Markov chain with steady-state probabilities Tr{b) ~ Tr{g) = 
0.5. Setting fco ~ 1. we can use Lemma [T] to show that 

|7rfc(s,=.s)-7r(.s, =s)|<2(l-/30^ (33) 

se{b,g} 

where /3; = 2r if r < 0.5, and /3; = 2 — 2r otherwise. Since 
there are N links with independent states, the above inequality 
indicates that after k transitions after time t, with probability at 
least (i — (1 — Z?;)*^)^, the state St+k satisfies St+k = s, where 
s can be any state in S. On the other hand, by Property [T] 
with probability at least 5, inequality (fTTI) with X = and 
s = St holds for any time-slot t. This implies that for any state 
s G S, there exists a set Ax^.s, ^Xt,s C T, such that for all 
I G ^Xt,s, inequality (fTTT i holds for Y = 1 and X = Xt, and 



E 



(34) 



where /iXt.s is defined in Section HV-AI 

Now, suppose at time t + k, for a given fc, the algorithm A 
chooses the schedule I that belongs to the set ^Xt,st+Jl- This 
happens with probability /iXt,st+fc (I*^ = I)- In this case, we 
have (p{t + fc) > — C- Assuming p > 2(, by the update rule, 
we have that with probability at least i, It+k = l^{t + k) = 
I. This and the discussion in the previous paragraph imply 
that for any channel state s and I G ^Xt.s, regardless of the 
initial state (sf,It), after fc time-slots, with probability at least 

'^In the context of this discussion, since we are focusing on by its 

definition, queue dynamics after time t are ignored, and thus j4xj_|.^. ,S(^f, = 



t + k ■ 
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PLniM) - - (1 - A)')^Mx„s(r = I), the chain 
will be at state {st+k,it+k) — Using this lower bound 

as the minimum transition probability in the expression for f3 
and replacing k with fco, we can show that (3 for the Markov 
chain {{st+i,It+i),i > 0} satisfies 

(sj):se5,ieAxt,B 



Finally, using the definition of and equalities ( [38] l. ( [39l ). and 
( |40| ). we can easily verify that we can choose such that 



(41) 



(l-/30'=''r7A'x.s(r = I) 



After studying for a Markovian channel model, we now 
return to finding an upper-bound for the dominant term given 
by Corollary ID Suppose arrivals are limited by a constant 
^maJE i-C-, ^lit) < Amax, I < I < N . Using this assumption, 
we can set A^^ = VN{A,nax + D,nax), where A^^ is given 
in the definition for . Using the same discussion leading 
to ( [32] | while not excluding the effect of Ae^, we see that A^^ 



I lo t ijzp wniie noi exciuuing me eiieci or Jie^, we see mai /ie^ 

T ^^(^ ^ (1 ^ E/ MXt,s(I'^ = I) contributes a ^/N term into -B^^,, and therefore, we can have 



4 ^'2 

ses 



> -6(1-2(1- Pi)''" 



(35) 

where, to obtain the last inequality, we have used ( |34] | and that 
\S\ = 2^ . Hence, we have obtained a lower-bound for /? given 
any fco. Next, we use this bound along with Lemma [T] to study 
the convergence of to $ Yj ■ 

Suppose TTk(-,-) is the distribution of the Markov chain 
{(sf_|_i, If+i), z > 0} after fc transitions, given Sj and If. 
From the definition of and $Yt' given in ( fTSl ) and ( l25T l, 
respectively, we have that 



1$ 



A' 



1 



i^T(Xi) 



E 

k=0 



E('rfc(s,I)-^(s,I))X,D(s,I) 



< 



(7rfc(s,I)-^(s,I) ViV||Xip, 



< 



k=0 

max 



fco2(l-/5)' < 



2ND,nax fco 1 



(36) 



i=0 ^ 

where the first inequality follows from (|2]i and ( l20l l. and the 
second inequality is a direct result of Lemma [1] Let 

^ r ln(4Af) i „ ^ n 



fco 



-ln(l-ft) 

1, r = 0.5 



which implies that (1-/3;)'^" < This inequality and ( l35T l 
further imply that /3 > |. Therefore, from 



16ND„ 



fco 



X(5 



we have that 
(37) 



We can obtain a similar upper-bound for | ^'y^ — ^'y^ I ^y 
considering separately two Markov chains corresponding to 
the pairs (st+k,lt+k) and (st+k+i,lt+k), fc > 0, respectively. 
Specifically, we can show that 

32ND^ax fco^ 



Using this inequality, the one in dJTl i. and the definition of 9, 
it is easy to see that we can set 



A1?> 



'IS' 



32ND. 



h 

max '^0 



(38) 



Using similar approaches, we can show that Ki^^ and £2 
can be chosen as 



and 



4fco 



ln(2) 



£2 



ln((l -(5)- 



£2 



(39) 



(40) 



In addition, suppose 



paoe 



(42) 



(43) 



This for instance is the case where the algorithm A chooses 
candidate schedules from a fixed set with equal probabilities. 
In this particular case, the distribution of I'' does not depend 
on Xj, and thus, can be assumed to be any positive real 

number Note that -B^^, depends on a specific implementation 
of the algorithm A, a topic that is not the focus of this paper. 

Recall that B ^ B, and K ^ K,. It follows from ([SB, dlD, 
dm, and (|43l) that 

B(^+2^M) = 0(^)+0(^^^). (44) 



ln(4jV) 



In addition, since — ln(l — /?;) > /?;, we have fcg < /j, ' 
which along with equality ( l44l l and Corollary |4] leads to the 
following corollary. 

Corollary 5. Suppose the state of each link is a two-state 
Markov chain with transition probability r and independent 
of the states of other links in the network. In addition, suppose 
arrivals are limited by a suitably large constant, Bi^^ satisfies 

Bpgj, = ^i ^pJse )' '^"^ P ^C- Then, assuming r < 0.5, the 
average queue-lengths satisfy the following: 

'N'^-%\n(N)y 



KKN 



N^(lii(N))^ ||a 



If r > 0.5, the same result holds except that the terms should 
be replaced with (1 — r)^. 

We are finally at a stage to study how delay scales according 
to various network- or policy-related parameters. Suppose the 
input rate is a = (Ai, • • • , Xn) strictly inside 9r, and consider 
the coefficients /3s j's corresponding to the rate a, as specified 
in Theorem [T] Let 9^ = maxgg^ X^ieJ Aj, where by the 
theorem we have 9^ < 9. Based on the definition of 9 and 9^, 
it is clear that the rate vector j-a belongs to the boundary of 
^r. Since the region 9r serves as the reference stability region, 
the rate fii = j-Xi, I < I < N, which is the l± element of 
the vector j-a, can be regarded as the effective service rate 
for the Zth link. This provides the motivation to define <; as 
<; = %■ and to consider as the effective load for each link. 



'Recall that we earlier in Section Ull-Al introduced A„ 
for the second moments of the arrival process. 



; as the upper-bound 
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Based on this definition, we have A; = <j/i;, 1 < Z < iV, and 
for e, as defined in Theorem [T] and used throughout previous 
discussions, we have that e = (1 — <;)9. From Little's theorem, 
we have that the overall average delay for each packet, denoted 
by D, is given by l/iJ2iLi ^i)'l2i<i<N -^i^ which along with 
the inequality '^^iXi > ||a|| and Corollary |5] leads to the 
following corollary. 

Corollary 6. Suppose the state of each link is a two-state 
Markov chain with transition probability r and independent 
of the states of other links in the network. In addition, suppose 
arrivals are limited by a suitably large constant, B^^^ satisfies 

B^j,^ = O ( ^pj^^ ), and p > 2(. Then, assuming r < 0.5, the 
overall average queue delay D satisfies 

^ ^ EkknXi ^ / jV4-5(ln(jV))2 s 

/^V5(ln(A0)^x 

where is the total service rate and is given by fX(^t) = 
^KKN l^i- U "f" > 0-5> the same result holds except that the 
terms should be replaced with (1 — r)^. 

Note that the format of the obtained delay-bound is similar 
to the average delay for the M/M/1 queue, which is -jj^^, 
with (, as the load and p. as the service rate. Remarkably, 
the corollary states that delay is polynomially bounded as the 
variables of interest, including the number of data flows N, 
take their limiting values. In particular, we see that delay is 
O(^) as the link variation rate r takes smaller values, and is 
Oyjsj as 5 0. In the next section, we consider both the 
throughput and delay performance of LM-RSP. 

VII. Joint Throughput-Delay Performance 

In this section, with the help of the corollaries provided 
earlier, we investigate the throughput and delay scaling as the 
variables of interest take their limiting values. As discussed 
earlier. Theorem [1] and Corollary [1] state that LM-RSP can 
stabilize a fraction of the capacity region F. Corollary [3] further 
shows that the policy can stabilize a fixed fraction Omin of 
the capacity region regardless of the number data flows N if 
the rest of parameters are fixed. However, as expected and 
inherently present in Theorem |2] the delay-bound increases 
with N . Specifically, Corollary |4] characterizes the dominant 
term in the delay-bound, leading to Corollary |6] that states delay 
is bounded by a polynomially increasing function of A^. 

An interesting trade-off occurs when parameters p and a 
take vanishingly small values. Recall that these parameters 
must be positive for continuity purposes. From Corollaries [T][3] 
we observe that as a and p take smaller values, the stability 
region QT is ensured to contain a region arbitrarily close to the 
region OumT or, for Markovian channels, the region Omin^. 
Corollary |6l on the other hand, shows that this comes at the 
price of increasing the delay -bound proportionally to 

A similar trade-off exists when the channel states be- 
come increasingly correlated. Specifically, as discussed in Sec- 
tion lV-C.Tl increasing channel correlation increases 9 and, thus, 
expands the stability region. We also discussed that channel 
correlation helps the policy compensate for a small 5. This 
is more explicitly stated in Corollary |3] which shows that for 
a particular Markovian channel process, as the link variation 



rate r decreases, the lower-bound region 9minX expands almost 
proportionally to (1 + (r + p)^-^)^^. In particular, in the 
limit of r — * and p ^ 0, for C = and any positive 
(5, we have 9min 1, and thus, throughput-optimality can 
be achieved, similar to the observation in [5]. However, as 
shown in Corollary |6] this makes the delay-bound increase 
proportionally to r^^ as r — > 0. 

Finally, we focus on the pair {C,S). Part (b) of Theorem [T] 
and also Corollaries [T][3] all state that C can directly affect 9 
through the term — = {1 ~ p){l ~ Q or, otherwise, 
the term (1 — (^). By Corollary |6] since for a given load 
factor the delay-bound increases proportionally to ^ as 
6* ^ 0, we have that the larger is C,, the smaller is 9, and 
the larger should be the delay. Therefore, increasing ( has 
negative effects on both throughput and delay. Recall that 
the parameter S is the least probability that the candidate 
schedules are within (^-neighborhood of the optimal schedules. 
Therefore, as is clear in the definition for 9, smaller values for 
5 decrease 9. Specifically, Corollary [3] shows that 9min, as a 
lower-bound for 9, decreases almost proportionally to as 
S approaches zero. Decreasing S has also an adverse efi^ct on 
the delay-bound since by Corollary |6] the bound can increase 
proportionally to ^ as J decreases. As the final remark, note 
that if it is possible to increase S, at the expense of increasing 
the complexity of algorithm A, it can be sufficient to make sure 
that S has the same order as r. This is an intuitive observation 
and a result of Corollary |3] which states that for Markovian 
channels, we may have 9min — jqqx^rjjf (1 ^ C)- For instance, 
when S — r, if the update rule is not used, the scaling of 
the capacity region is r(l — C) whereas using LM-RSP can 
ensure 9mm > ^(l — C)^ which implies a significant throughput 
improvement especially when r ^ 0.5. 

VIII. Conclusion 

In this paper, we have studied the stability region and delay 
performance of a linear-memory randomized scheduling policy 
LM-RSP for networks with time-varying channels. LM-RSP 
uses an update rule along with a randomized algorithm that 
with probability at least S finds a candidate schedule vector that 
is within (-neighborhood of optimality. The complexity of LM- 
RSP depends on the complexity of the randomized algorithm 
and, in particular, may be linear. We have proved that LM-RSP 
can stabilize a scaled version (fraction) of the capacity region 
and quantified the corresponding scaling factor as a function 
of the parameters in LM-RSP and the limiting behavior of 
rate changes due to channel variations. Furthermore, we have 
provided an average delay-bound for general ergodic channel 
processes. For a particular class of Markovian channels, we 
have shown that the average delay is C'(^), as r 0, 
where r is the link (individual channel) variation rate, and is 
bounded by a polynomially increasing function of the number 
of data flows. In addition, for this class of channels, we have 
shown that a minimum fraction ^^(1 — C) of the capacity 
region can be stabilized. Our results also indicate that while 
the minimum fraction decreases Unearly as (5 ^ 0, the delay 
may increase as j^, and therefore, the effect of S on delay may 
be more severe than the one on the stability region. The results 
in this paper are promising and motivate further research as 
they indicate that even when channels are time-varying, using 
randomized policies can help stabilize a predictable fraction 
of the capacity region, in networks with limited computation 
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power and memory resource, while assuring a polynomially- 
bounded delay. 

Appendix 

Proofs of Theorem[T]and Theorem|2] 

The proofs use several lemmas that are provided in a separate 
appendix at the end of the paper. 

Proof of part (a) of Theorem\l\ Since a is strictly inside 9T, 
there should be a rate ai inside F such that a = 9ai. By the 
definition of F, it is easy to see that there should exist non- 
negative constants (3'^ j's such that for all s e 5, X^iei P'si^^ 
and ai = Ese5 '^(s) Eiei /5s,iD(s, I). Using this equality 
and setting /3s. i = OP'^ i, we see that for these choices of /3s j's, 
a satisfies jZTT i. and e as defined by e = ^ — maxsg^ Tliiei 
is positive, as required. □ 

Proof of part (b) of Theorem \J\ We use a K-step drift analysis 
to prove part (b) of the theorem. The main difficulty here is 
to properly use the properties of LM-RSP in the drift analysis. 
Consider the following Lyapunov function: 

N 

1=1 

We can write a K-step drift as follows: 

A{K)t = nV{Yt+K) - V{Yt)\Yt] 



and 



E 

k=Q 
A'-l 



E[F(Yt+fc+i)-^(Yt+fe)|Y4 



k=0 
A'-l 



= J2 E[2 Xt+fc(At+fc - Dt+fc) +2 Xt+fcUt+fc + 

<5i,/c '52. fc 

{At+k - (Dt+fc - Vt+k)){At+k - (Dt+fc - Ut+k)) \Yt]. 

(45) 

Based on the above expression, there are three main summa- 
tions, corresponding to 6i^k, S2.k, and S^^k, respectively, each 
of which should be upper-bounded appropriately. To upper- 
bound the summation over 62. note that by the definition of 

U(i), U,{t) < D^ax, and if X,{t) > Dmax, then U^{t) = 0. 
Therefore, 

K-l A'-l 

E E[<52,fc|Yi] < ^ NDl^^ = KNDl^,^. (46) 

fe=0 k=0 

As for the summation over 5z.k, we have that 

A'-l A-1 

nszA'^t] < nAt+kAt+k\Yt]+nBt+kT>t+k\Yt] 

k=0 k=0 

<Knm']+KNDi^^. 

The heart of the proof, however, lies in deriving an upper- 
bound for the summation over Si^k- It requires Lemmas |2]- 
I2I which are listed at the end of the proof of theorem. First, 

consider the following straightforward observations: 

fc-i 

^Xt+kAt+k\Yt] < E[XtAt+fc|Yt] +E[^ At+,At+fc|Yt] 

1=0 



A'-l 



Yn^tAt+k-Xt+kT>t+k\Yt] 

k=0 

A-1 

= i^Xta- ^E[X,+fcD,Vfe|Yi] 

fc=0 

A-1 

+ Yn^t+k{'D*t+k-'Dt+k)\Yt]. (48) 



fc=0 



Since by the assumption in the theorem a is strictly inside 
dV, part (a) of the theorem holds. Specifically, there exist non- 
negative constants /3sj's such that equality (IZTT i holds, and e 
as defined in part (a) is positive, i.e.. 



e = 9 — max > /3s i > 0. 
se5 ^ ' 



(49) 



To use the results of Lemmas 2-6, suppos^ ei — €4 — e'^^ = 
^DZ-fh £5 = ^cil^^tf' and 67 = |, where e is given 
by g9). We set 

K ^2ina^iKi^,„Kit,Dmax-,--,K^t^) = K„ (50) 

v e 

where the last equality follows from the definition of given 
in In addition, let 62 = e'^ = ^d^J!!^^ ^3 = ^3 = 
h-rr—TriA^, and B = B^i ^„ where°!Bf5,, is defined in 



(51) 



Lemma |5] These choices imply that 



B = Bf , 



= Xta + fc||a||2, 



(47) 



where Bf- is defined in dsTl ). 

With these choices, we can put together the results in ( |45] ), 
( |46l ), ( |?7] i, ( |48] l to show that 

A(if )t < + ifE[|l A||2] + KNDl,, 

A-1 

+ 2 ^ E[Xt+fc(At+fc - Dt+fc)|Yt] 

fc=0 

< + A-E[||A|p] + 2X2||af 

A-1 

+ 2{KXta - ^E[Xt+feDt+fc)|Yt] 

fe=0 

< 3X/VI?L. + ^E[||A||2] + 2X2||a||2 

A-1 

2(ifXta- ^E[Xt+feD*+fe|Yt]) 

A'-l 

-h 2 5^ E[X,+fc(D*+, - D,+fc)|Y,]. (52) 

k=0 

Using Lemmas H] |6] |7] we can find an upper-bound for the 



'"Note that th e sequence of e^'s, 1 < i < 5, here is different from the ones 
I Section IV-Al 
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above, leading to the following for |lXf|| > 50 

A{K)t < SKNDl^^ + XE[||A|n + 2K'\\a\\' 

2K^NDl^^ + 2KD,naxVNei\\Xt\\ (53) 

- 2if ^7r(s)(l - ^/3,^i)XiD*(X,,s) 
ses lex 

2/^ I t^./at, II V II I 1 



+ 2min \^KC2 + K^C^ + XViVeellXtH + Kpa5 
+ /^T(X.)(c' + ti)vl,^^), 

if^C^ + ifViVe^llXj + A'T(X0(1 - $^;)^ , (54) 

where C2, C3, and cq are given in Lemma |6] and C3 and eg 
are given in Lemma [T] 

After a few simple algebraic steps, we obtain 

A(A')f < KCi + A'^Cs 



2K \ l-maxV/3. 
\ lei 

/ pCkllXf" 



+c' + V*Y;,i-'i>^.) )T(xo 



+ 2KVN{€iD^ax + te)WXtl (55) 

where C4 and C5 are constants defined by C4 = 2C2 + 
iV(i2^,, + ZDl,,) and C5 = 2C3 + 2||af + 2iVi^^,,. 

With our choices for e^'s, 1 < i < 5, and AT, it is easy to 
verify that 

V e 

ilDrnax + ^6 < ^ ■ (56) 

In addition, using the inequality in ( |20] i. the assumption that 
AT > Arif^ with 67 = |, and the fact that min{f(x)) = 
—max(~f{x)), we can show that 



where is the indicator function and 

Ck,b = 2i^B||a|| + i^E[||A|n + K{K - l)\\a\\^ 

+ K^NDl,,. (58) 
Now, we take the expectation of A{K)t with respect to the 
distribution of Yf, which leads to 

A'{K)t^m\xt+Kf]-m\xtf] 

Xtll < -A'eE[ ||Xt|ll||x,||>B ] +Ck + Ck,b. 

Considering the above inequality for times i + jK for i E 
{0, . . . , AT — 1} and j E {0, . . . , J — 1}, and summing over i 
and j, we obtain 

^ = E (E[I|X,:+,7A'|P] -E[||X,|p]) 

i=0 j=0 1=0 

_ff-l ,7-1 

^-^^EEe[I|X.+,a-||1,[x.+,,.||>b] 

+ JK{Ck + Ck,b)- 
Since norm is a non-negative function, from the above, we have 
that 

A'-l ,7-1 

E EJE[I|X.+,k||1||x.+,.-|I>s] 
i=o i=o 



>„,ax(l-^-<'-i^*?„*?,) 



> 61-67 



Using ( I20I 1. (|49] l. (l55T l. and the previous inequality, we have 

A(^^)t < KCi + if^Cj. _ ^i^(e 



e TV 



^ 4 

where eg = (eiA'maa; + ce)- Using ( |56] l. for ||XJ > _B, we 
obtain 

A{K)t < CK-K^\\Xt\\, 

where £ = and 

Ck ^KCi + K^Cs. (57) 

Using the assumption that the second moments of the arrival 
process are finite as specified in (O, we can generalize the 
above inequality for all ||Xf|| as 

A{K)t<CK-K^ |1XJ1||x,||>b+Ca- ,sl||Xt|l<B 

< -K^ |lXt||l||x,||>B+C^A- + CA-,B, 

"Note th at b oth Lemma |6] and Lemma [T] provide an upper-bound for the 
last term in (52), and we use the minimum upper-bound in our analysis. 



i=0 



Using the fact that E[||Xt|| l||Xt||<B] < B, and letting t 
i + jK and T = JK — 1, we obtain 

f]E[iix,ii] < (T + i)B + -i- Y.n\\M'] 



t=o 



■^(Ca + Ck.b) 



(59) 



Since the first and second moments of the arrival process are 
finite, for a fixed A', the summation on the righthand side of 
( |59b can be bounded by an appropriate constant Cg. Hence, we 



must have Eilo JE[||Xj||2] < Therefore, we have 

T 

VE[||Xt||] < {T + 1)B+^C,-, + ^{Ck + Ck.b)- 

f^o K^ ^ 

Finally, dividing by T + 1 and letting T 00, by assuming 
J ^ 00, we obtain 

T 



limsup-l-^E[|lXi|l] <B 



Ca + Ck,b 

Ik 



< 00, (60) 



which completes the proof of part (b) of the theorem. □ 

Proof of Theorem^ To start, first note that YldLi-^ii^) — 
•\/iV||Xt||. We can use this inequality, the one in ( l60b , and 
equality ^ = along with the definitions for Ck and Ck,b, 
given in (|57| | and (l58l l. respectively, to show that 

T N 

Iimsiip-1-^[^E[X,(<)]1 <S(VA^+2-||a||) 



t=o 1=1 



N 



{2NAI 



16 6-^NDt.^K 



+ 4(5 'pa\/N{A,r,ax + Dmax)K\, 



(61) 



where according to the proof in part (a), K ~ K^ and B — 

see ( [soi l and dSTT i. respectively. To obtain the above inequality. 
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we have also used the fact that if a G F, then a/ < Dmax, 
I <l < N, and hence, ||a|| < \/ND„iax- 

Note that K depends on N and S. Therefore, to complete 
the proof, it suffices to show that the expression in the large 
brackets in ( |6T1 ) is 0(^KNS^^^ with respect to variables S 
and N. This easily follows since by ( l23T l. ^/NpaS^^iy^^ < 1, 
completing the proof. □ 

Appendix 
Lemmas 

Here, we often use subscripts to denote time dependencies. 
We use 1 to represent a vector with all its elements equal to 
one and use Ig to denote the indicator function for the event 
e. Where required, we use underlines to show realizations of 
rv.'s, e.g., in our analysis, Xt+m denotes one realization of the 
random vector Xf+m. 

Lemma 2. For all values of t, s G S, and m > 0, we have 
that 

E [Xt+m+i (D* (Xt_|_„j_|_i , s) -D*(Xt+„„s))|Yt] <Ci, 
where Ci = ^D,nax\M + ND^^^^. 

Proof. By the definition of D*(X, s), as provided in (flOt . we 
have 

Xt+m+l (j^* s) — D* (Xt+,„, s)^ 

= Xt+„j I D* (Xt+„j+i, s) — D*(Xt+„i,s) 



+rn+l 



X 



■t-\-ni 



D*(Xt+™+i,s)-D*(X: 



< (^Xt+„j+i — Xt+mj (^D*(Xt+„i+i, s) — D*(Xt+„i,s 
^D*(Xt+„i+i, s) -D*(Xt+m,s 



< D 



max-^t-\-7n 



1 + NDl 



where the last inequality follows from the assumption that 
transmission rates are bounded by D„iax- Taking the expected 
value of both sides and noting that At+m is independent of 
Yt, proves the Lemma with the given Ci. □ 

Lemma 3. For any two queue-length vectors Xi and X2, and 

(81,82) £ S X S, the following holds: 

X2(D*(X2,82)-D*(X2,8i)) 

-Xi(D*(Xi,82)-D*(Xi,8i)) 

< (X2-Xi)(D*(X2,S2)-D*(Xi,si)). 

Proof. The proof easily follows from the definition of 
D*(X,8) given in (Hol). □ 

Lemma 4. Suppose a is given by (1271 ), where non-negative 
constants Ps,i's satisfy X^igi/^s.i ^ 1 for all s £ S. For any 
positive ei, if K > Ki_^-^, then we have 

K-l 

Ax. = KXta- nXt+kT>*t+k\Yt] 

k=0 

< K^ND^^^ + KD„,ax^ei\\Xt\\ 

Ky^TT{s){l - J2 /3s,l)XtD*(Xt, 8), 



ses 



where Ki^^-^ is defined in Section \ni-B\ 

Proof. First, note that the following holds using the definition 
for D* (X, 8) and the assumption that D„iax is the global upper 
bound for individual rates. 

Xt+fcDt+fc = maxXt+fcD(8t+fc,I) 

fc-i 

> maxXtD(st+fc, I) - max ( ^ Df+jD(s4+fc, I)) 



> niaxXtD(st+fc, I) - kND. 



4=0 

2 

max • 



(62) 



In addition, for a given X, by the assumption in the lemma 
and the definition for T(X) given in ( fTSI l. we have 

Xa- T(X) 

= X ^ 7r(s) /3s,iD(s, l)-Yl '^(s)XD* (X, s) 



ses iGi 



se5 



-^^(8)(l-^/?,,i)xD*(X,s) 
se5 lei 
<-^7r(s)(l-^/3,,i)XD*(X,8), (63) 



se5 



where the last inequality follows from the definition of 
D*(X,8) implying that XD(s,I) < XD*(X,8) for all I e X. 
Now, we can use (l62T i to show that 

Ax, < K^NDl,, + KXtS. 

- E E[Xt^l,(,+fc)^,D*(Xt,s)|Y/ 

A:=0 se5 

= K'NDl,, + KXteL 

-^XtE[^ l,(t+fe)=,|Yt]D*(Xt,s) 

+K Y 7r(s)XtD* (Xt , 8) - A' E ^i^)^t^* i^t,s) 



ses 



ses 



K-l 



+KJ2 ('TIS) - E[-i Y W)=s|Y*])x,D*( 



X.,s) 



se5 



fc=0 



+K{Xta~T{Xt)), 



lex 



where in the last equality, we have used (fTST l. 

When K > i^i,^, the inequality (|6]l holds for e — ei. Using 
this, inequality (|63] l. and that transmission rates are bounded by 
Dmax, we obtain the inequaUty in the lemma for K > Ki^^i, 
as required. □ 



Lemma 5. Suppose arbitrary positive integer K and arbi- 
trary positive real numbers ti and €3 are given. Let A^.^ — 
y/NDjyiax + ^e2> where A^^ is defined by Fact\l\ and let 
^^,i3 ^ ^^'^(^Ms' -^^ea)' where C = KA^^ and B^^,^ and 
are defined by Property^and Fact^ respectively. Then, 
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the following inequality holds ;/ ||Xt|j > 

K-2 

5]e[x,(d,+„-(i-p)d;+J|y,_ 

m— 

K-2 

- ^ Ex. [M^t+m - (1 - p)D;+,„) \Yt 

m=0 

< KD^ax^(2Ke2 + ((1 + 263)''^ - l)) ||Xt||. 



Proof. Recall that the notation represents the expectation 
assuming that the policy does not have updated queue infor- 
mation after time t. Hence, while in the first summation in 
the lemma, we are taking a typical expectation considering 
all dynamics of the system, in the second summation, we 
are taking the expectation assuming that at any time t + m, 
TO > 0, the scheduling policy updates the schedule vector It+m 
using the old queue information at time t, and therefore, by 
setting Xt-|_,„ — Xj, to > 0. In the following, to distinguish a 
particular realization of a random variable, we use underlines, 
and in the subscripts, we remove dependencies on time. For 
instance, I[ represents one realization of II^.^. 

For K> 1, let Xf = {X.+JfL,, sf = {s.+J^,, 1^^ = 
{IJ'+JfLi, and If = {lt+t}f=i. For fc > 1, the following 
equalities follow easily from the update rule properties and 
using the fact that the arrival process is an i.i.d process and 
independent of the channel process. 



k—l rr,k—l -rk—l 



p(st+fc|xf,sr\it 

p(I[+,lx^s^Ip'=-^I^^Y,) 

=-p(ir+fciXt+fc,st+fc), 
p(I,+felx^s,MI■^I^^Yo 

= p(It+fc|Xt+fc, st+fc, Ij^j., It+fc-i)- 
Let Wt+i, i > 1, be defined as 

Wi+i — (Xf+i, Sf+i, If+i), 

and Wf be {Wt+i}iLi- Hence, Wt+i contains all r.v.'s 
associated with time t + i, and Wf is the collection of Wf+^'s 
over K time-slots. We also use the same variable W 



K 



to 



represent (Xf , sf , ' , If ) that essentially contains the same 
set of rv.'s. as does {Wt+i}iLi- Using conditional probabiUties 
and the above equalities, we have 

p(Wf |Y0 = 

K 

n |Xt+i_i, st+fe_i, If+i_i)p(st+i|sJ"\ Yt) 

i=l 

p{Vt_^.,\'Xt+^,St+^)p{It+^\'Xt+^, St+i, Ij+,, If+j_i) 

(64) 

Similarly, for the case where the policy does not have 
updated queue information, let ij'^ — and if — 

{It+i}fLi^ where I[_^j and It+i are the candidate schedule and 
the updated schedule at time t + i, respectively, based on the 
assumption that after time t, old queue information at time t 
is used by the policy. Let Wf+i be 

Wt+i = (st+i, I[_^j, It+i), 



and Wf = {'^t+i}i^i- We use the same variable Wf to 
represent (sf , i^'^, if ). Using the same approach to simplify 
p(Wf |Yt), we can show that 

K 

Px.(Wf |Yt) = n (p(st+.|sr\ Y*)p(ir+JXi,st+,) 



p{It+^\'Xt,St+^, I[+j, It+^-l)j, (65) 

where the notation px^{■), as defined in Section [V-AI is used 
to emphasize that Wf is a set of r.v.'s with the assumption 
that the policy does not have the updated queue information 
after time t. 

To continue, let Xf = (Xf+i, . . . , Xf+x) and 
7^(Xi,/^,i,J = {X^'; ||Xi - Xill < 

||X,-X.-ill <ie„* = 2,...,/0. 

Note that, as mentioned in the beginning of the proof, Xj 
represents one realization of X^^^. By assumption, A^^ — 
VNDmax — A^2^ ^iid hence, by Fact[Tl regardless of Xf+i_i, 
Sf_|_i„i, and It+i_i, we have 

P^W^t+i — ~^t+i~l\\ > ^£2! Xt+i-li St+j-1, It+i-l) 

<p(||At+,_i|| > A,,-^/nD 

max ) 

< £2. (66) 

Using the above inequality and the union bound, we have 

p{X.f in{y.uK,A,,)\Yt)<Kt2. (67) 

By definition, Xf e 7^(Xt, if, implies ||Xt+,-Xt|| < 
KA^^, for I < i < K. Therefore, if Xf e 7^(Xf,^^:,A,J 
and ||XJ > = max(Bf;,3,B£,J, where C = KA,„ 

by Property |2] we have 

J2 = I|Xt+„ s,+,) - p(ir+, = I|X,, s,+,) I < £3. (68) 

lei 

for all I e I and St^i E S. Similarly, assuming-for a particular 
reali7ation-I[+^ = = I[ and It+i-i = it+t-i = I^-i, by 
Fact 121 we have 



P{lt+i — I|Xt+i, St+,;, I[_^j, It+i_i] 



-p{it+i = i|Xf,st+j,i 



t + i: It + i-lJ 



<e3, (69) 



for I e all s e 5, and all I[ and I^_i in X. 

We are now ready to consider the summation terms in the 
lemma. First, for a given s^ — {s^}^i and I^ = {lilfLi^ 
define the function 5(s^,I^) as 

A'-l 



i=0 



Using this function, we define the rv. ^(Wf ) as 

K-l 

g(Wf ) = g(lf , sf ) - ^ X, (Bt+m - (1 - p)D;+„) 



m — 



K~l 



Xi(^D(sf+,,i,Ii+„J - (1 - /?)D(si+„i+i,It+„j) 



m— 



(71) 



Setting as one realization of Wf , which contains the 

realization sequences X^, s^, , and I^, where, e.g., V'^ 
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represents one realization of we have 
E[5(Wf)|Y,] 

= 5]p(Wf =W^|Y,).9(W^) 



E 
E 



p(Wf =W^|Yt)g(W«) (72) 



p(Wf =W^|Yt)5(W^)- (73) 



In above, we have separated two cases by considering whether 
or not Xf^ e TZCX-t, K, Ae^)- We have also used under-lined 
variables to highlight realizations of the their associated random 
variables. Since for all 

\g{W'')\<Knraa.VN\\Xt\\, (74) 
by (l67T l. we have 

m<K^DrnaccVNe2\\Xt\\. ^ (75) 

To simpUfy note that when Xf e n{Xt,K, 4^ J and 
> S^^eg, inequaUties (|68]l and (|69]l hold. Using these 
inequalities and (|64] i. we see that for each realization W^^ of 
the random vector Wf^, we have 



p(wf -w^'iY^)=n 



p(x.t+i =Xi|Xt+i_i =Xj-i,s 



t+»-l — 



Pi st+i = sM 1 = s' \ Yt 



■t+i 



t+i 



6.-1 



It+i — li Xt, St 



^ — I[,It+i-l 



6.(1.) 



(76) 



where 6i-i(ID 6i(Ii)> 1 < * < -f'^. that are specified as 
functions of I[ and Ij, respectively, in general depend on 
with the property that Ekgi l6i-i(lDI < ^3, and |6.(IJ| < 

£3- 

Considering ( |76] ), we see that there are 2^^ terms in the 
expression for p(Wf^ = W^|Y(). Each term is obtained 
by including either the error terms 6i-i and or their 
corresponding probability expressions, for all i, where 1 < i < 
K. This implies that the summation in ( |73] ) can be decomposed 
into 2^^^ sub-summations. The main sub-summation is the 
one where no error is present. Using i65[ and ( l76b , we have 

= J2 Px. (Wf = w'^'lYOsd^, s^) 

K 

E n = xjxt+j-i = Xi_i, 



st+j- 1 = 1 , It+i- 1 = I.- 1 ) j ■ (77) 
Note that in above the summations are nested. By the definition 
for [•], we have 

EPx.(Wf =w''|Yt)g(I^,s^) = Exj5(lf,sf)|Yt]. 



This equality along with the inequalities in ( l66b and ( l74l l yields 
|S™-ExJ<?(I^,s^)|Yt]| 

<ifA/iV||Xt||A„a.(l-(l-e2)'^) 

max 

VNWXtl (78) 

Now, we consider the effect of errors. Let Ef, ,, be the 
sub-summation corresponding to the term with ki errors of the 
form ^223-1, where ii < Z2 <,...,< i^i, and with fc2 errors 
of the form form ^2;^, where li < I2 <,■■■,< lk2- Using (l74b 
and that |6i(Ii)| < £3. we have that 

|Se| < 

5]p(st+i|Yt)^p(ir+i|Xt,st+i) 

p(it+i|Xt,st+i,i;+i,it)--- X!p(st+»N^^Yt)■■■ 
E ^^.i-idlj • • • Y p(it+i\Xt,st+i,ii^i,it+i-i] 

p(W|Xt, st+K, ir+if , W-i)i^™a.V]vif iiXtii 

Ifc6{lK.lK-l} 



K 



E 

'eK(Xt,A:,i, 



p(Xt+i|Yt) (^p(Xt+,|Xt+i_.i,st+i_ 



~l,It+i- 

< Xi^„,,V7V(e3)'=n2e3)'l|Xt||, (79) 

where in the above, summations are nested, and to make 
the probability expressions shorter, we have shown only rv.'s 
and removed their corresponding equality expressions. For 
example, we have used I^'^j to represent = I[. To obtain 
the last inequality, we have considered the summations from 
the innermost to the outermost, and we have used the fact that 
the summation of probabilities over all possible choices adds 
up to one, the summation over the error term of the type 6jj-i 
contributes a multiplicative factor of £3, the summation over the 
error of the type ^2/ contributes a multiplicative factor of 2e3, 
and the fact that for any given and s^, the last summation 
is less than one. 

The inequality ( |79] l shows that the obtained upper-bound 
does not depend on the order of i/s or Ij's. Since there are 
number of sub-summations with ki errors of the type 
^2i-i and with k2 errors of the type £^2i, considering all possible 
values for fci and fc2, in particular the case where ki = k2 = 
that is accounted for by S.,„, and using ( |78] l and (|79] l, we obtain 

|(l73)-Exj5(l'',s'')|Yt]| < 
KDma:.y^[Ke2 + ((1 + eafil + 2e3f - l)) ||Xt 

Combining and dSO]), we have 

< 



(80) 



E[g(Wf)|Yt]-Exj5(lf,sf)|Yt] 

KDma.^(2Ke2 + ((1 + 2e3)2^ - l)) |lXt|l, (81) 

which by the definitions of g{lf^,sf) and g(Wf^), given in 
( iTOb and ( ItTI i, respectively, is the same as the inequality in the 
lemma except that in the left hand side of dSTl ), K should be 
replaced with if — 1. If we consider if — 1, in all previous 



17 



Steps, we require |lXf|| > ^^^^3^^ But B^^ ^,^ > b[^^^,^\ On 
the other hand, the right hand side of dSTl l is an increasing 
function of K. Therefore, ||Xt|| > B^^ is sufficient to have 
the inequality in the lemma, completing the proof. □ 



Lemma 6. Suppose arbitrary positive real numbers 62, £3, £4, 
and £5 are given, and K—1 > max{Ki^^^, K2'^^^), where Ki ,^^ 
and k'^I^ are defined in Section UlI-B] TfUXf || > B^ where 
Bf^ is defined in Lemma |5] the following inequality holds: 

K-l 



k=0 



^E[Xt+fe(D,V,-Di+fc)|Yi] 

< KC2 + K^Ca + K^/Nee\\Xt\\ + Kpa6-^\\Xt\ 
+ ifT(XO(c' + ^^y^*^ 



where C2 — 2i5 ^(1 — i5)Ci with Ci defined in Lemma\2\ C3 
VNDraaxM +26-^1 -6)Ci+pa6-^{\\eL\\+VN{A^acc 
Dmax)), C defined in i22i . and 



ee ^D„,a J 6-\2K 62 + {{l + 2e: 



\2K 



1)) 



£4 + 2£5(5-i 



K 



Proof. We first focus on each individual term in the summation 
in the lemma. Let Af denote the event that (fTTT i holds at time t 
for X = X(t), s = s(t), and I'' = V{t). Using Property □ and 
the fact (O still holds when D'(t - 1) is replaced by W{t), 
we obtain 

Ad,^, ^E[Xi+fc(D:+,-Di+fc)|Yi] 
<,5C' E[Xi+fcD,Vfc|Yt,At+fc] 
+ pa5 ^[WKt+k\\ I YuKt+k] 
+ (1 - <5)E[Xi+fc(D,V,. - Di+fc)|Yi, A^+,] 

= 5 C' E[Xi+fcD,V,|Yi] +pa5 E[||X«+,|| | Y*] 



+ (1-(5)E 



(1 - p)d;+,_i 



(1 - p)T,[^k-i - ^t+k 



Yt,A: 



t+fc 



(82) 



where C, is defined in (|22| |. In the above, we have used the fact 
that the current and the past queue-lengths, or the past chosen 
rates, are independent of the event At+k (or its complement), 
and that the event At+k occurs with probability 6. 

Using ( IT4] | and adding and subtracting Ht+k-i, we can show 
that 

Ad.+. <<5C' E[X,+fcD*+,|Yt] 



+ (1-(5)E 



^t+k\i->t+k 



D 



t+fc-i 



+ Dt+fc_i-(i-p)D;+fc_i 

pa(l-5)E[||Xi+fc|l I Y,,A,V] 



Y,,A 



+ pa<5E[||Xt+fc|l I Y^]. 
Similarly, since A^^^, is independent of Xf+fc, by adding and 



subtracting Dj^j,_j^, we obtain 
Ad,^, < 5 C' E[Xi+fcD,Vfc|Yi] 



+ (1-(5)E 



Yt,A^+, 



YuK+k 



+ (l-,5)E[Xt+fc(Dt+fe_i 

- (1 - p)d;+,^i) ' 

+ paE[||Xt+fe|||Yt], 
which combined with Q yields 

Ad,+. <<5C' E[X,+feD*+,|Y,] 

+ (1 - <5)E[Xi+fc(Di+fc_i - (1 - p)D;+,_i)|Yi] 
+ (1 - ^)E[X*+,.(D:+, - D:+,„i)|Yt] 
+ (1 - (5)E[(At+fc_i - Dt+fc_i + \Jt+k-i) 

(D*+,_i-Di+fc_i)|Y, 
+ paE[||Xt+fc|||Yt] 

+ (1 - (5)E[(Xt+fe_i)(D*+,_i - Di+fe_i)|Y,] . (83) 



By induction and summing over fc, from ( l83l l. we obtain 

^E[X,+fe(D*+,-D,+fe)|Y,] 



fc=0 



= E E (l-'5)^-^<JC' E[Xi+™+iD*+„+i|Y,] (84) 

m— 1 

K-m-1 



m— I— 1 



K-2 K-m-1 



m— i—1 



K-2 K-m-1 



Dt+„-(l-p)D;+J|Yt 
D 



't+m+l -^t+m; I'^t 



D! 



E E (i-'^rJE 



m— i—1 



K-2 K-m-1 



[^t+rn ^ Df+„i) I Yf 



+ P^Y. E (l-^y-^EillXi+^+illIYi 



m— 1 



(Xt(D,*-D, 



(85) 

(86) 

(87) 
(88) 

(89) 



In the sequel, we find an upper bound for each summation 
term in the above. First, note that similar to ( |62] |. we have the 
following inequality: 

E[Xt+„j+iDj%,„_|_i|Yt] < 

maxE[XtD(st+„+i,I)|Yt] + ViV(m + l)i?„a,||a||.(90) 
Using this inequality and that <5X]ta™~^(l ^ < ^' 



18 



have 



fc-2 



rn—0 sG5 

+ {K~ 1)C' 7r(s)XtD*(Xt, s) + if2v^D„,a.|la|l 

<C'(if-i)E 



se5 



(^[if^ E ls.+„+,=s|Y,] -4s))x,D*(Xi,s) 



where 7/f_m„i belongs to the sequence 7 that is defined 
in Section IIII-BI We first find an upper-bound for A*-'^. Let 
yext A (Y4,St+„j,Sf+,„+i). Using Lemma[3l we have 

(si,S2)e52 X ^ ' 

P(si,S2) 

p(Xt+™+i = X|Yr*) x(d*(X,S2)-D*(X,si) 
^ E E S2)p(X,+,„+i = X|Yr*) 

(si.S2)e5^ X 

(Xt(D*(Xt,S2)-D*(Xt,si)) 

+ (X - X0(D*(X,S2) - D*(Xt,si)) 



+ {K- 1)C E[XtD*(Xt,s)] + K^VNDmaxM 

< KeiWXtW^D^ax + A'C'T(Xt) + K^v^Dma.M, 

(91) 

where the last inequality follows from the assumption that K > 
Ki^^i, the inequality (' < 1, the definition in ( fT6l l, and that 
Dmax is the transmission rate upper-bound. 

To derive an upper-bound for (|85] l. we can use (|5j and the 
inequality ^^^"^^^{^ — Sy < S^^ {1 — S) to show that 

K—2 m m 

< r 1(1 - <5) ^ e[( ^ A, - ^(D, - uo) 

m=0 i=0 (=0 

(Di+,^-(l-p)D;+,J|Yi 

K-2 



Replacing the distribution over the queue-length vector with the 
distribution over arrivals and transmission rates, which leads to 
the same expectation, and letting 

p({A,}r=o,{Dj™oiYr*) = 

p{{M+^,}T=o - {Ajr=o,{Dt+^}"o = {Dj™oiYr)> 

we can rewrite the above inequality as 



p(si,S2)Xt(D*(Xt,S2)-D*(Xt,si)) 

(si,s2)e52 



Ep(x 



t+m+l 



= XIYf 



X 



+ 



+ S-\l -5)Y IE[X,(D,+,„ - (1 - p)D;^.„J|Yi] 

K-2 

< S-\l - <5) ^ (m + l)(^/iVA„a.|la|l + (1 - p)Dl,.,N) 



E E p(si'S2) 

(si,S2)GS= {Aj™o,{DJJ^o 

p({AJIlo:{DJIlolYr) 



m— 

K-2 



+ 5-\l-5) Y IE[X,(Dt+„ - (1 -p)D;+„J|Yt]. (92) 

Since by assumption ||Xt|| > we can use Lemma|5]to 

replace the expectation in the above with an expectation of the 
type lExt[']^ leading to 

K-2 

+ 5-\l -S)Y IExJX,(Di+„, - (1 - p)D;+„J|Y,] 

m=0 



N 



2Xe2 + ((l + 2e3f^-l) ||X 



(93) 



We now focus on ((861). Let D*-^ = D*(Xt+i,St); hence, 
T)^ is the optimal rate corresponding to the queue-length 
vector at time-slot t + 1 but the channel state at time t. Using 
this definition and Lemma |2] we can show that 

K-2 



(EA7 - E(S« - D*(X„si)) 

1=0 i=0 

< ^P(st+m+i = s|Yt)XtD*(Xi,s) 
- ^p(st+m = s|Yt)XtD*(Xt, s) + (m + l)Ci, (95) 

where the term (m+ l)Ci, with Ci defined in Lemma |2l is an 
upper-bound for the second bracketed summation in the above. 
Note that in the above, Xt+m+i represents one realization of 
the vector Xj+m+i and is determined by X( and the sequences 
{AJ™o and {DJ™^._^ 

Now, let Si^ = J2m=o iK-ra-i, for which we have Si^ < 
KS~^{1 — 6). Using this definition and the assumption that 
K -1> K^^X, we obtain 

K-2 

I E 



m=0 



t+m+l (Df^^^i 



i^p(st+„+i = s|Yt)XtD*(Xt,s) 

sG5 

Y 7K-m-lE^(^)^*°*(^*'^) 



m—0 sGS 
K-2 



m=0 



seS 



D:+„o|Yi 



K-2 



sg5 \ t > m=C 



(gK) ^ 7J<'-m-ilst+„+i=s|Yf -7r(s) 

V 7 / m—0 



K-2 



m=0 



< ^7K-™-i(Ci+E[Xi+™+i(D*+„+i-D:^„)|Yi| ), 



A*/ 



(94) < Ke5S-\l - S)VND,nax\\Xt 



(96) 
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We can obtain a similar inequality as the one in ( |96] l with 
St+m+i is replaced with St+m, which along with the results 
from dllli, and yields 

(HHl < {K + K^)S-\l - 6)Ci 

+ 2Ke56-\l-6)VNDraaA^t\\- (97) 
Remaining terms that should be upper-bounded are ( [87] l. 
dSSl l. and ( [89l ). Using the triangle inequality and the fact that 

m\\Xt+,n+i - XtW \Yt] f < E[||Xi+,„+i - XJ2|Y,], we 
can show that 

E[||Xt+„,+i|| \Yt] < ||Xt|| + v/E[||Xt+^+i-Xt|P \Yt] 
< \\Xt\\ + \^mv/lE[||A||2] + (m + l)||a|| 

ax • 

Hence, 

dSl < paS-^EWXtW 

max + Anax)), (98) 

where Amaa; is defined in (|3]l. Using the same methods, we 
can easily show that 

(183 < K5-\l - <5)Ci, m < S-'^D„,ax\\Xt\\. (99) 
Combining the results from ([H-dHlll, dlB, 
and (l99l l. we obtain the inequality in the lemma, as required. 

□ 

Lemma 7. Suppose arbitrary positive real numbers ej, 63, ant/ 
£4 are given. Suppose K > Ki ,,' , where Ki ,,' is defined in 
Section Ull-B] Then, if WKA] > , , where bH , is defined 
in Lemma |5J the following inequality holds: 

5^E[Xt+fc(D:+,-Di+fc)|Yt] 

fc=0 

where C[^ — Ci with Ci defined in Lemma\2\and 

4 = amax(2Ke'^ + ((1 + 2e;,)2^ - l) + e^) . 

Proof. The proof is a similar but a simpler version of the proof 
given for Lemma |5] □ 
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